ComponentType: add as_val method to convert to tagged representation#12496
ComponentType: add as_val method to convert to tagged representation#12496pchickey wants to merge 8 commits intobytecodealliance:mainfrom
Conversation
alexcrichton
left a comment
There was a problem hiding this comment.
Personally I'm not overly thrilled about extending the Lift/Lower/etc traits given the complexity of testing them and the sheer breadth of impls required, but this is overall relatively minor in terms of complexity. Before landing mind also expanding the PR description with some rationale for the expected use cases?
Bikeshedding wise I might say to_val instead of as_val since "as" implies "cheap" typically and this definitely isn't cheap. I'd also bikeshed the signature a bit to use StoreContextMut<T> rather than impl AsContextMut to avoid the extra indirection of impl Trait where possible.
The 'static bounds popping up on futures/streams is fine, those are only really practically usable with 'static types anyway, so I don't think that'll be an issue. The specifics of as_val for futures/streams I think will want to be re-worked a bit too to avoid manually creating structs, but I can take a closer look once this is closer to landing.
Similar to futures/streams I'll need to think a bit harder about the try_as_resource_any method added here for resources. I forget exactly where we are on clone/copy/etc on those and how it affects what we document in terms of guarantees and whatnot.
e2db39e to
cb3e478
Compare
which requires adding 'static bounds on the ComponentType, Lift and Lower impls
cb3e478 to
2bbc956
Compare
ffd19c7 to
c15f664
Compare
This PR adds the method
fn to_val<S>(&self, store: StoreContextMut<S>) -> Result<Val>to wasmtime'sComponentTypetrait. This is a first step in a bigger project to provide better debugging facilities for wasmtime component model execution.Context
Wasmtime separates component model values into two separate worlds:
ComponentTypetrait, plus optionally theLiftandLowertraits (required to be used in arguments and return values of import functions, respectively) which provide wasmtime the means to convert to/from each Rust type and its canonical ABI representations. To use this representation for an import function wasmtime users usewasmtime::component::Linker::func_wrapand friends, and on an export function usewasmtime::component::Instance::typed_funcand friends.wasmtime::component::Valenum. To use this representation for an import function, wasmtime users usewasmtime::component::Linker::func_newand on an exportwasmtime::component::Instance::func.Prior to this PR, users have to choose whether to opt into the typed representation or the tagged representation based on what their application needs, and we (the wasmtime developers) focused most of our efforts on the typed representation because that ends up being the right choice for most embedders.
Users of the typed representation do so almost exclusively through the use of
wasmtime::component::bindgen!, a proc macro which generates Rust structs and enums corresponding to component model types it parses from wit, and delegates to the#[derive(ComponentType, Lift, Lower)]derive proc-macros in those generated types (which themselves also are provided by the wasmtime crate). This bindings generation typically takes place in library code, e.g.wasmtime-wasi,wasmtime-wasi-httpand related crates, andbindgentakes a variety of options that influence exactly what sort of Rust representation to use for wit types - theComponentTypederive macros permit flexibility in various ways. But, at any rate, to my knowledge every "serious" embedding of wasmtime (rough definition: engineers ship it to prod at dayjob) is written in Rust and usesbindgenexclusively or very close to it.On the other hand, the tagged representation is useful for library code that exists independently from a given wit description. The library for a human-friendly serialization format for component model values
https://crates.io/crates/wasm-waveexports some traits which wasmtime provides impls for in terms ofwasmtime::component::Val. Wave ends up being useful for features inwasmtime-clilikewasmtime run --invoke ..., which I introduced in #10054. The C API, which is used for bindings to other languages, also uses the Val representation, because C APIs and the bindgen proc macro are oil and water.My goal is to create better debugging tools for wasmtime component model programs, providing wasmtime embedders and cli users with the capability to inspect, and eventually record/replay or modify component model values. Like the wave crate and invoke integration, debugging tools benefit from being independent of a given wit description. Debugging tools must integrate with wasmtime embeddings regardless of whether they use the typed or tagged representation to implement import funcs and call export funcs, and since most embeddings are making substantial use of
bindgen!in public and private crates against public and private wits, my approach is to provide a bridge where a debugging library can operate on just the tagged representation of values, using a new mechanism built into all typed representations to convert to tagged representation. Further mechanisms in wasmtime to hook into library code using the typed representation will come in future PRs, see #12532 for work in progress.Converting typed to tagged
For every value in a typed component model value representation (i.e. any value of a type which has an
impl ComponentType), there also exists an equivalent component model value inwasmtime::component::Val(tagged) representation. Prior to this PR, there was no mechanism to convert a typed value to a tagged one.This PR adds a method
to_valtoComponentTypeto convert to a Val:The design choices in this signature are:
&self, notself, because the reverse conversion does not exist, and in the context of adding debugging hooks to existing code using the typed representation, we don't want to destroy the typed representation in order to observe it.Val, which means making a structural clone of the value. For a witlist<u8>this ends up having some fairly ugly overhead: the typicalVec<u8>, which implsComponentType, Lift, Lower, converts to theVal::List(Vec<Val>)variant, where each vec element is then aVal::U8(u8)variant. Other variants, likeVal::Record,Val::Variant,Val::Enum, andVal::Flagsend up usingStringrepresentations of the various wit identifiers used for record rows, variant and enum tags, and each flag value. This requires substantially more memory to represent, and is substantially less efficient to traverse!Valhas further caveats where it does not provide the same safety guarantees as the typed representation for resource, future, and stream representations, see https://docs.rs/wasmtime/latest/wasmtime/component/struct.ResourceAny.html for information on the requirements for dropping a resource, which also apply when in a leaf node of aVal. WhenResource<T>::to_val(&self)is invoked, FIXME FIGURE OUT WHAT HAPPENS HERE AND WHAT THE USER NEEDS TO DO TO HANDLE IT. DOES VAL NEED Adrop{,_async}(self,StoreContextMut)METHOD? and put whatever i learn in the proper docsimpl AsContextMutfor the store, but Alex suggested I switch to useStoreContextMut, which then requires the function to be generic on the type in the store. For this type variable I useSinstead of the usualTbecauseTis often already in use in the impl.The
ComponentTypederive macro fills out a definition of to_val mechanically - this was the only aspect of the PR that was subtle to figure out. All of theimpl ComponentTypefor Rust types provided by wasmtime get a straightforward mechanical implementation.TODO before this PR can land: